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ABSTRACT 



The Teacher List Validation Study was conducted to evaluate 
the quality of the Teacher Listing Record (TLR) , a form used during the 
Schools and Staffing Survey (SASS) to obtain a list of teachers in each 
school. The SASS is a set of four inter-related surveys of public and private 
elementary and secondary schools, districts, teachers, and administrators 
conducted by the National Center for Education Statistics (NCES) . For the 
first component of the study, 100 private and 100 public schools were 
selected for reinterview to determine whether the TLRs had been completed 
accurately. In the second component of the study, an additional 100 schools 
and 100 districts were reinterviewed to determine whether the TLR or the 
school questionnaire provided a more accurate account of teachers. Data 
suggest that public schools were more accurate listing teachers than were 
school districts. The types of errors usually made were characterized. 

Private schools incorrectly included nonteachers and incorrectly excluded 
teachers, although the instances were few in each teacher/nonteacher group. 
Both public and private schools were more accurate in listing teachers using 
the TLR than the SASS school questionnaire. Part-time teachers were most 
often omitted, and the types of nonteachers most often included erroneously 
were librarians and prekindergarten teachers. Two attachments describe the 
study sample and teacher/nonteacher categories for the report. (Contains 6 
graphs and 14 tables.) (SLD) 
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Foreword 
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contained in these documents and to promote the sharing of valuable work experience and 
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undergo vigorous NCES publication review and editing prior to their inclusion in the series. 
Consequently, we encourage users of the series to consult the individual authors for citations. 

To receive information about submitting manuscripts or obtaining copies of the series, 
please contact Suellen Mauchamer at (202) 219-1828 or U.S. Department of Education, Office 
of Educational Research and Improvement, National Center for Education Statistics, 555 New 
Jersey Ave., N.W., Room 400, Washington, D.C. 20208-5652. 
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Summary 



A. Purpose 

We conducted the Teacher List Validation Study to evaluate the quality of our 
Teacher Listing Record (TLR). This form is used during the Schools and 
Staffing Survey (SASS) to obtain a list of teachers in each school. 

We designed the study to be primarily qualitative in nature. Also, because we 
selected a non-random sample, we conducted no statistical tests on the results. 

B. Major Findings 

1. Reinterview and Reconciliation of the TLR 

• The data suggest that the public schools were more accurate listing 
teachers than their corresponding school district (Local Education 
Agency, or LEA). 

• Public schools and LEAs often omitted part-time and specialized subject 
matter teachers from the TLR. (Attachment B contains a complete 
description of the teacher groups.) 

• LEAs often incorrectly listed guidance counselors, while public schools 
most often erroneously listed librarians and speech therapists as 
teachers. 

• Although the private schools incorrectly included non-teachers and 
incorrectly excluded teachers, the instances were few in each 
teacher/non-teacher group. 

2. TLR vs. School Questionnaire 

• The data suggest that both the public and private schools were more 
accurate listing teachers using the TLR than the school questionnaire 
(SASS-3X). 

• The public and private schools often omitted part-time teachers when 
reporting their teacher count using the school questionnaire. 

• The types of non-teachers most often included in error on the SASS-3X 
by the public and private schools were librarians and pre-k teachers. 
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C. Recommendations 

• Continue to use the public schools, rather than the LEAs, to obtain teacher 
counts using the TLR. Although the schools were not completely accurate, 
they were more accurate at listing teachers than their corresponding LEA. 

• The schools appeared to provide a more accurate teacher count using the 
TLR, rather than the school questionnaire. Even though this was the case, 
the design of the TLR and the instructions need additional research. 

We used the results of this study to revise the TLR for the 1993-94 SASS. It 
contained more specific instructions regarding who should and should not be 
included on the list. 

The Center for Survey Methods Research (CSMR) has implemented a 
program of cognitive research to improve the TLR. The results of this 
research will be available in the fall of 1994. 



n. Methodology 

During the SASS, the TLR is used to obtain a list of teachers in each school. The 
Census Bureau then selects a sample of teachers from each school for the teacher 
survey portion of SASS. 

The study was prompted by the fact that both the 1987-88 and 1990-91 SASS estimated 
different numbers of teachers using the TLR compared with the school questionnaire. 

In the 1987-88 SASS, the schools, on average, reported fewer teachers on the TLR 
compared to the school questionnaire. In the 1990-91 SASS, the schools reported 
different numbers of teachers on these two forms. The TLR is always mailed to the 
schools at the beginning of the fall semester. The school questionnaire (in the 1987-88 
and 1990-91 SASSs) was mailed to the schools in the beginning of the spring semester. 

The study contained two components, each with different objectives. The objectives 
for component 1 were to: 

1) determine if the schools filled out the TLR per our instructions (i.e., the 
instructions printed on the form), 

2) determine if the schools listed eligible in-scope teachers, 

3) determine if the school districts (Local Education Agencies, or LEAs), rather 
than the public schools, could provide more accurate listings of teachers. 
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The objective of component 2 was to determine whether the school questionnaire or the 
TLR produced a more accurate count of teachers in the school. The assumption was 
that the counts from the school questionnaire are more accurate. 

A general objective for both components was to find out if certain types of 
teachers/non-teachers created problems for the schools when computing the teacher 
counts (i.e., teachers were systematically missed or included incorrectly and/or their 
definitions were misunderstood). 

A. Component 1; Reinterview and Reconciliation of the TLR 

In component 1 we compared the teacher count obtained from the TLR completed 
by the public school to the one completed by the LEA for that school. For 
private schools we compared the teacher count on the TLR completed by the 
school to the teacher count as reported by the school in the 1991-92 PSSl. We 
selected the 200 reinterview cases (100 public and 100 private) with the largest 
percent differences between the counts. 

1. Sample Design 

The sample for component 1 consisted of 300 private schools, 290 public 
schools, and 254 LEAs. (Some LEAs had more than one sample school.) 
This sample did not overlap with the component 2 sample. 

Attachment A contains a more detailed description of the sample. 

When we received approximately 85 percent of the TLRs, we began 
selecting our reinterview samples. (We couldn’t wait until we received all 
of the TLRs to select the reinterview sample because it would have been too 
late to begin the reinterview.) We selected 100 public schools (with their 
corresponding LEA) and 100 private schools. 

Note: The schools (and LEAs, for public schools) were eligible for 

reinterview only if they sent in a completed TLR. If either a public 
school or its LEA did not return a TLR, they were not eligible for 
the reinterview sample. 



1 The PSS refers to the 1991-92 Private School Survey. This survey collects data from 
private schools on student enrollment, student race/ethnicity, school programs, number 
of teachers, and type/location/affiliation of school. 
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a. Public Schools 

We matched each public school TLR with its corresponding LEA 
TLR. We then selected the 100 public schools with the highest 
difference rate as defmed below: 

L = number of teachers reported on the LEA TLR only 
S = number of teachers reported on the school TLR only 
B = number of teachers reported on both TLRs 

difference rate = (L + SI 

(L + S + B) 

The difference rates ranged from .87 to 0. The 100 public schools 
that we selected for reinterview ranged from .87 to .11. 

b. Private Schools 

We selected the 100 private schools with the highest difference rate 
(positive or negative) as described below: 

S = total number of teachers reported on school TLR 
P = total number of teachers reported in the 1991-92 PSS 

difference rate = (S - PI 
S 

The difference rates ranged from 23.5 to 0. The 100 private schools 
that we selected for reinterview ranged from 23.5 to .18. 

Note : Unlike the public schools, we could not consider the actual 

teachers listed in developing this rate because the PSS only 
provided counts - not names. 

2. Procedures / Timing 

In mid-November, 1992, a TLR was mailed to each private school, public 
school, and each public school’s LEA (254 LEAs in all - 290 
questionnaires). Telephone nonresponse follow-up for these questionnaires 
began in the regional offices in mid-December. Closeout was at the end of 
January, 1993. (This timing was consistent with past SASSs.) 

Reinterview began in mid-February, 1993 and ended at the end of March. 
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The reinterviewers were given no formal training. They were instructed to 
read the Reinterviewer Instructions Memorandum, and use their overall 
experience and knowledge on conducting reinterviews. The field 
representatives that were used were familiar with conducting reinterviews. 

Note : The original TLRs were edited in the regional offices. Contrary to 
what is done normally (i.e., in past SASSs) we decided that if the 
regions found errors on the TLRs, they should not contact the 
schools to correct them. Instead, they documented the errors on the 
TLRs. We hoped to find out the reasons for the errors during the 
reinterview. 

a. Public Schools 



Of the 100 public schools selected for reinterview, 50 were conducted 
by personal visit, and 50 by telephone. The reinterview method was 
assigned by ranking the 100 cases in order from largest difference rate 
to smallest, and then alternating, beginning at the top of the list, by 
personal visit and telephone. 

We instructed reinterviewers to contact only the school (not the LEA) 
for these reinterviews. 

i. 50 personal visit cases 

For the 50 personal visit cases, the original school respondent 
completed another TLR, thinking aloud as he/she filled out the 
form. Our goal here was to determine how the respondent 
interpreted our instructions. 

Once this was done, we instructed the reinterviewer to compare 
the reinterview list with the original list filled out by the 
respondent, and reconcile any differences. The reinterviewer 
was also instructed to determine why the LEA reported certain 
teachers that the school did not. 

ii. 50 telephone cases 

For the 50 telephone cases, the respondent did not complete 
another TLR. Instead, the reinterviewer was instructed to 
reconcile the differences between the TLR filled out by the 
school and the one filled out by the LEA. 
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We mailed back to the school a copy of the original TLRs 
completed by the school and the LEA, along with a letter 
describing the study and that someone from the Census Bureau 
would contact them regarding the reconciliation. This was done 
so that the respondent would have the necessary information 
when the reinterviewer called. 

b. Private Schools 

Of the 100 private schools selected for reinterview, 50 were 
conducted by personal visit, and 50 by telephone. These were 
assigned by ranking the 100 schools from the largest school (in terms 
of size, not difference rate) to the smallest. We determined the size 
of the school by the number of teachers in the school as reported on 
the original TLR. The larger 50 schools were assigned personal visit 
reinterviews, while the smaller 50 were assigned telephone 
reinterviews. We did this because it would be easier to conduct a 
reinterview for a smaller school over the phone than it would for a 
larger school. The largest school in the telephone sample contained 
1 1 teachers. 

Note : We did not conduct a reinterview for the public school 

telephone reinterview cases. We reconciled the differences 
between the school TLR and the LEA TLR. Therefore, it 
was not necessary to purposely include the smaller sized 
schools in the public telephone reinterview sample. 

i. 50 personal visit cases 

The reinterviewers conducted the 50 personal visit cases the 
same way as the 50 personal visit cases for public schools. The 
reinterviewer instructed the school respondent to complete 
another TLR, thinking aloud as he/she filled out the form. 

Once completed, the reinterviewer compared the reinterview 
TLR with the original TLR filled out by the respondent, and 
reconciled any differences. 

ii. 50 telephone cases 

The 50 telephone cases were done the same way as the 50 
personal visit cases. The respondent completed another TLR 
over the phone. The reinterviewer then reconciled the 
differences between the original TLR and the reinterview TLR. 




: c 
JL o 



7 



B. Component 2: TLR vs. School Questionnaire 

In component 2, we compared the teacher count reported by the school on the 
TLR to the teacher count reported on the school questionnaire. No LEAs were 
involved in component 2. For the reinterview, we selected 200 cases (100 public 
and 100 private) with the largest differences between these counts. 

1. Sample Design 

The sample consisted of 300 private schools and 290 public schools. This 
sample was selected the same way as the component 1 sample. (See 
Attachment A.) 

We selected the component 2 reinterview sample in two phases. We did 
this because the school questionnaires came in slower than expected. When 
we received approximately 50 percent of the school questionnaires, we 
selected phase one of our reinterview sample. After we received about 90 
percent of the school questionnaires, we selected phase two of the 
reinterview sample. We selected a total of 100 public schools and 100 
private schools. We selected the public and private school reinterview 
samples the same way. 

We selected the 100 public schools and 100 private schools with the highest 
difference rate (positive or negative) as described below: 

T = number of teachers reported on Teacher Listing Record (TLR) 
X = number of teachers reported on school questionnaire 
(SASS-3X) 

difference rate = fT - XI 
T 

The difference rates ranged from .98 to 0 for the public schools (.98 to .05 
for the 100 public schools selected for reinterview), and from 2.0 to 0 for 
the private schools (2.0 to .07 for the 100 private schools selected for 
reinterview). 
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2. Procedures / Timing 

Component 2 began by mailing a TLR to each public school and private 
school in the Component 2 sample. The timing for this mailing was the 
same as Component 1. (They were mailed in mid-November, 1992. 
Telephone nonresponse follow-up began in the regional offices in 
mid-December, and closeout was at the end of January, 1993. This timing 
was consistent with past implementations of the SASS.) 

At the end of February we mailed a SASS-3(X), school questionnaire to 
each school in the Component 2 sample. Telephone nonresponse follow-up 
began at the end of March, and closeout was at the end of May. (This 
timing was consistent with past implementations of the SASS.) 

The SASS-3(X) was a field test version of the school questionnaire. It was 
being tested in preparation for the 1993-94 SASS. It contained questions 
about the characteristics of a school. Included in the questionnaire were 
questions regarding the number of teachers in the school. We were 
interested in these teacher count questions only. 

Reinterview began at the beginning of May, and ended in mid-June. 

Each public school and private school reinterview in component 2 was 
conducted by telephone. We mailed back to the school a copy of the 
original TLR and school questionnaire filled out by the school, along with a 
letter describing the study and that someone from the Census Bureau would 
contact them regarding the reconciliation.. This was done so that the 
respondent would have the necessary information when the reinterviewer 
called. 

The 100 public school and 100 private school respondents did not complete 
another TLR (as was done in component 1). Instead, the reinterviewer 
reconciled the differences between the teacher count reported on the original 
TLR, and the teacher count reported on the school questionnaire. 
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C. Determining the Actual Count of Teachers 
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Before we could do the data analysis, we had to determine the actual count of 
teachers in each school (i.e., the "truth" count). We used this count as the basis 
for our comparisons. We did this by: 

• reviewing the instructions on how to complete the TLR and school 
questionnaire 

• reviewing the TLR completed by the school (and LEA, for public schools) 

• reviewing the reinterview TLRs completed by the schools (component 1 only) 

• reviewing the school questionnaire completed by the schools (component 2 
only) 

• reviewing the explanations and notes provided by the reinterviewer regarding 
the list of teachers on the TLRs, the count of teachers on the TLRs, and the 
count of teachers on the school questionnaires 

• using the definitions and guidelines that NCES has set up regarding (for our 
purposes) who is, and who is not a teacher. 

D. Limitations 

Certain aspects of the study limited us to what we could and did find out. 

First of all, this study was designed to be qualitative in nature rather than 
quantitative. We weren’t trying to get specific numbers on how many teachers 
were erroneously missed or non-teachers that were erroneously included. Rather, 
we attempted to find out the types of teachers/non-teachers that the schools 
included or excluded in their counts. For that reason, we did not find out, for 
example, the exact number of librarians that were being counted as teachers. We 
did, however, find this out at the school level (i.e., how many schools were 
including librarians in error). 

Second, we wanted to find out reasons why the schools excluded certain teachers 
and included persons that should not have been included. Unfortunately, the 
reinterview and reconciliation did not gather adequate reasons. Most of the 
respondents simply said they "forgot about that person" or "I thought this person 
should/shouldn’t be included." Other respondents simply didn’t provide reasons. 

Third, our initial sample was not unbiased or random. We purposely wanted in 
the sample those schools that historically reported (based on past SASSs) 
inconsistent teacher counts using the TLR and the school questionnaire. 
(Attachment A describes the sample in more detail.) The reinterview samples 
also were not unbiased or random. 
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Because of the way we selected the samples, statistical testing on the results 
would be inappropriate. Although we do say that certain counts are greater than 
others, there is no statistical evidence that this is so. We merely "eyeballed" the 
numbers. 



HI. Results 

The results in this report are shown separately by component (1 and 2). Within each 
component we show results for public cases and private cases. The component 1 
public school cases are also shown separately by telephone reinterview and personal 
visit reinterview. We do not have separate sections for the private school reinterview 
telephone and personal visit cases. Other than the method of reinterview, the 
procedures were the same for these cases. There was also no major procedural 
difference between the private telephone reinterview cases and the private personal 
visit reinterview cases. 

We compared the counts obtained from the TLRs (components 1 and 2) and school 
questionnaires (component 2, only) to the actual count of teachers in the school. We 
used only those cases where we could determine a "truth count". (See table below.) 
For public schools, we also compared the count of teachers from the CCD! to the 
truth count. For private schools, we compared the count of teachers from the PSS to 
the truth count in component 2 only. We already know that these counts will differ 
for the cases in component 1 because we selected this sample based on the large 
difference between the TLR count and the PSS count. 

We discovered that the CCD and PSS counts were not very accurate. This was 
probably due to the fact that they were not current counts. The CCD counts were 
from the 1990-91 school year. The PSS counts were from 1991-92. 

We also kept track of the different types of teachers that the schools (and LEAs, for 
public schools) incorrectly EXCLUDED in their TLR and school questionnaire 
counts, and the different types of non-teachers that were incorrectly INCLUDED in 
their counts. Attachment B contains the 19 different groups that we used to classify 
these types of teachers and non-teachers. We used all completed reinterviews for this 
analysis. (See table on next page.) 



2 Common Core of Data (CCD) - This file contains public school data provided and 
updated by the state each year. LEAs provide the schools’ data to the state. We used 
the teacher counts from the 1990-91 CCD file for the 1993 TLVS. 
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The table below describes the response rates. 
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Component 1 


Component 2 


Public 


Private 




Phone 


PV 3 


Phone 


PV 


Public 


Private 


Total Sample 


50 


50 


50 


50 


100 


100 


Complete 

Reinterviews 


50 


49 


47 


48 


100 


98 


Refusals 


0 


1 


2 


2 


0 


0 


Unable to Contact 


0 


0 


1 


0 


0 


2 


"Truth" Count 
Determined 4 


50 


49 


47 


48 


88 


88 



A. Component 1: Reinterview and Reconciliation of the TLR 

1. Public School - Telephone Reinterview 

Of the 50 LEA TLRs, 14 contained the correct number of teachers in the 
school and 27 were within ± 5 percent of the truth count. Nineteen school 
TLRs contained the correct count, and 34 were within 5 percent of the truth 
count. The LEA and school counts (the exact matches and the counts 
within 5 percent) appear consistent. The data suggest that these counts are, 
however, greater than the CCD counts (6 exact matches, 16 within 5 
percent). See table 1 and graph 1 on next page. 



3 PV = personal visit reinterview 

4 We used these cases for the percent difference analysis. 



Table 1 
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Distribution of Public Schools/LEAs by Percent Difference 
from Truth Count 



Source 
of Data 


0 % 0<P<5 


No. of schools/LEAs 
5<P<10 10<P< 15 


P> 15 


total 


LEA TLR 


14 13 


11 


5 


7 


50 


school TLR 


19 15 


6 


4 


6 


50 


CCD count 


6 10 


11 


7 


16 


50 



• P = the percent difference from the truth count 



Graph 1 



Comparison of Teacher Counts 

Public Schools, telephone (50 cases) 
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2. Public School - Personal Visit Reinterview 

Only 3 of the 49 LEA TLRs contained the correct count of teachers in the 
school. Twenty were within 5 percent of the truth count. Fourteen school 
TLRs contained the correct count, while 32 were within 5 percent of the 
truth count. The data suggest that the school counts (both the exact match 
and the count within 5 percent) are greater than both the LEA counts and 
the CCD counts (5 exact matches, 14 within 5 percent). The LEA counts 
appear fairly consistent with the CCD counts. See table 2 and graph 2. 



Table 2 Distribution of Public Schools/LEAs by Percent Difference 
from Truth Count 



Source 
of Data 


0 % 


0<P<5 


No. of schools/LEAs 
5<P< 10 10<P< 15 


P> 15 


total 


LEA TLR 


3 


17 


11 


6 


12 


49 


school TLR 


14 


18 


8 


1 


8 


49 


CCD count 


5 


9 


9 


6 


20 


49 



• P = the percent difference from the truth count 



Graph 2 



Comparison of Teacher Counts 

Public Schools, PV (49 cases) 




% Difference 
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3. Public School - Total 



Of the 99 LEA TLRs, 17 contained the correct count of teachers in the 
school, while 47 were within 5 percent of the truth count. Thirty-three 
school TLRs contained the correct count, and 66 were within 5 percent of 
the truth count. The data suggest that the school counts (the exact match 
and the count within ± 5 percent) are greater than both the LEA counts and 
the CCD counts (11 exact matches, 30 within 5 percent of the truth count). 
The LEA counts appear greater than the CCD counts. See table 3 and 
graph 3. 



Table 3 Distribution of Public Schools/LEAs by Percent Difference 
from Truth Count 



Source 
of Data 


0 % 


0<P<5 


No. of schools/LEAs 
5<P< 10 10<P< 15 


P> 15 


total 


LEA TLR 


17 


30 


22 


11 


19 


99 


school TLR 


33 


33 


14 


5 


14 


99 


CCD count 


11 


19 


20 


13 


36 


99 



• P = the percent difference from the truth count 



Graph 3 



Comparison of Teacher Counts 

Public Schools, total (99 cases) 




% Difference 



88S LEA vs. truth ■■ school vs. truth CCD vs truth 
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Of the 99 public schools in the component 1 reinterview sample, we 
discovered, through the reinterview, that the LEAs and schools incorrectly 
excluded the following types of teachers the most: (See Attachment B for a 
complete list and description of the groups of teachers and non-teachers.) 



Table 4 Types of Teachers Excluded on TLR by LEAs and Public 


Schools 








Number of 


Number of 


Teacher Categorv 


LEAs 


schools 


• general full-time 


30 


22 


• part-time 


21 


15 


• specialized subject matter 


17 


15 


• special ed. 


10 


10 


• long-term sub 


10 


6 


• itinerant 


9 


5 


Note: Only one public school (and it’s LEA) missed a homebound 


teacher. It was picked up during the reinterview. 


This was the 


only instance in the study where a school (public or private) 


mentioned they had a homebound teacher. 




We discovered that the LEAs and schools incorrectly included the 


following types of non-teachers the most: 






Table 5 Types of Non-teachers Included on TLR by LEAs and Public 


Schools 








Number of 


Number of 


Non-teacher Category 


LEAs 


schools 


• other non-teacher 


18 


11 


• librarian 


10 


18 


• speech therapist 


10 


18 


• guidance counselor 


14 


9 


• principal / asst, principal 


6 


3 


• other school staff 


5 


4 
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4. Private School - Total 

Seventy-three of 95 school TLRs contained the correct count of teachers in 
the school, while 76 were within 5 percent of the truth count. The data 
suggest that there is no major difference between the telephone cases and 
the personal visit cases. See table 6 and graph 4. 



Table 6 Distribution of Private Schools by Mode of Reinterview and 
by Percent Difference from Truth Count 

Source No. of schools 

of Data 0 % 0<P<5 5<P<10 10<P<15 P> 15 total 



telephone 


39 


0 


2 


4 


3 


47 


PV 


34 


3 


4 


2 


5 


48 


total 


73 


3 


6 


5 


8 


95 



• P = the percent difference from the truth count 



Graph 4 
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Overall, the 95 private schools in the component 1 reinterview sample 
didn’t seem to have a lot of problems listing teachers (that is, not nearly as 
many as the public schools). Of these schools, we discovered that the 
schools incorrectly excluded the following types of teachers the most: 



Table 7 Types of Teachers Excluded on TLR by Private Schools 



Teacher Category 


Number of 
original 
TLRs 


Number of 
reint. 
TLRs 


• general full-time 


5 


3 


• subject matter 


4 


2 


• specialized subject matter 


2 


4 


• part-time 


1 


3 


• other teacher 


2 


1 



We discovered that the schools incorrectly included the following types of 
non-teachers the most: 



Table 8 Types of Non-teachers Included on TLR by Private Schools 

Number of Number of 
original reint. 


Non-teacher Category 


TLR$ 


TLRs 


• principal / asst, principal 


4 


6 


• pre-K 


2 


7 


• librarian 


2 


5 


• other non-teacher 


2 


3 



B. Component 2: TLR vs. School Questionnaire 
1. Public Schools 

Fifty-five of 88 school TLRs contained the correct count of teachers in the 
school, while 68 were within 5 percent of the truth count. The SASS-3X 
count was correct for 12 cases, while 26 were within 5 percent of the truth 
count. The data suggest that the TLR counts are greater than the 3X 
counts, and both the TLR counts and the 3X counts are greater than the 
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CCD counts (2 exact matches, 16 within 5 percent of the actual count). 
See table 9 and graph 5. 
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Table 9 Distribution of Public Schools by Percent Difference from 
Truth Count 



Source 
of Data 


0% 0<P<5 


No. of schools 
5 <P< 10 10<P< 15 


P> 15 


total 


3X quest. 


12 14 


20 


13 


29 


88 


school TLR 


55 13 


13 


4 


3 


88 


CCD count 


2 14 


12 


12 


48 


88 



• P = the percent difference from the truth count 



Graph 5 



Comparison of Teacher Counts 

Public Schools, Comp. 2 (88 cases) 
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Of the 100 public schools in the component 2 reinterview sample, we 
discovered that the schools incorrectly excluded the following types of 
teachers the most: 
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Table 10 Types of Teachers Excluded on TLR and School 
Questionnaire (SASS-3X) by Public Schools 



Number of Number of 



Teacher Category 


3Xs 


TLR$ 


• part-time 


18 


10 


• general full-time 


15 


5 


• special ed. 


3 


9 


• specialized subject matter 


2 


6 


• subject matter 


0 


5 


• itinerant 


2 


3 


• Chapter 1 


2 


3 



We discovered that the schools incorrectly included the following types of 
non-teachers the most: 



Table 11 Types of Non-teachers Included on TLR and School 
Questionnaire (SASS-3X) by Public Schools 

Number of Number of 



Non-teacher Category 


3Xs 


TLRs 


• librarian 


11 


4 


• other non-teacher 


10 


4 


• pre-k 


4 


3 


• principal / asst, principal 


4 


2 


• guidance counselor 


4 


2 


• speech therapist 


2 


4 



Note : Three 3X questionnaires included persons with teaching and 

non-teaching duties. Two were guidance counselors and one 
was an other school staff person. The 3X instructed the 
respondent to do this, while the TLR instructions did not. 
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2. Private Schools 



Of the 88 school TLRs, 5 1 contained the correct count of teachers in the 
school, and 55 were within 5 percent of the actual count. The SASS-3X 
count was correct for 33 cases, while 35 were within 5 percent of the truth 
count. The data suggest that the TLR counts are greater than the 3X 
counts, and both the TLR counts and the 3X counts are greater than the 
PSS counts (15 exact matches, 21 within 5 percent of the truth count). See 
table 12 and graph 6. 



Table 12 Distribution of Private Schools by Percent Difference from 
Truth Count 



Source 
of Data 


0% 


0<P<5 


No. of schools 

5 <P< 10 10<P< 15 P> 15 


total 


3X quest. 


33 


2 


14 


9 30 


88 


school TLR 


51 


4 


13 


11 9 


88 


PSS count 


15 


6 


14 


10 43 


88 



• P = the percent difference from the truth count 



Graph 6 



Comparison of Teacher Counts 

Privaie Schools, Comp. 2 (88 cases) 




% Difference 
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Of the 98 private schools in the component 2 reinterview sample, we 
discovered that the schools incorrectly excluded the following types of 
teachers the most: 
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Table 13 Types of Teachers Excluded on TLR and School 
Questionnaire (SASS-3X) by Private Schools 





Number of 


Number of 


Teacher Category 


3Xs 


TLRs 


• part-time 


13 


17 


• general full-time 


6 


10 


• subject matter 


1 


4 


• Chapter 1 


2 


3 


• specialized subject matter 


0 


4 


• itinerant 


3 


0 


We discovered that the schools incorrectly included the following types ■ 
non-teachers the most: 



Table 14 Types of Non-teachers Included on TLR and School 
Questionnaire (SASS-3X) by Private Schools 

Number of Number of 


Non-teacher Category 


3Xs 


TLRs 


• pre-k 


13 


1 


• librarian 


6 


4 


• other non-teacher 


8 


0 


• principal / asst, principal 


5 


2 


• other school staff 


4 


1 


• guidance counselor 


4 


0 



Note : Twenty-two 3X questionnaires included persons with teaching 

and non-teaching duties. (Two of the 22 contained two different 
types.) Twenty-one were from group (1) - principals, one was 
from group (2) - guidance counselor, and 2 were from group (6) 
- other school staff. The 3X instructed the respondent to do 
this, while the TLR instructions did not. 
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TLVS Sample 



Attachment A-l 
June 3, 1994 



The public school sample was selected from the public school universe file that was going to 
be used for the school phase of the 1993 SASS. (The 1993 SASS was ultimately postponed 
until 1994.) Each public school’s LEA was selected as well (for component 1, only). The 
private school sample was selected from the private school universe file that was current at 
that time (August 1992). 

Before selecting the public and private school samples, we deleted schools in certain states 
from each of the universe files. These states had high field costs. The states are listed 
below. 



States dropped from TLVS 



• 


Alaska 


• 


Minnesota 


• 


North Dakota 


• 


Arizona 


• 


Mississippi 


• 


Ohio 


• 


Arkansas 


• 


Missouri 


• 


Oklahoma 


• 


California 


# 


Montana 


• 


Oregon 


• 


Colorado 


# 


Nebraska 


# 


Texas 


• 


Hawaii 


# 


Nevada 


• 


Utah 


• 


Idaho 


• 


New Mexico 


• 


Washington 


• 


Kansas 


• 


New York 


• 


Wyoming 


• 


Louisiana 









After we deleted the schools in the states listed above, we selected the samples. We selected 
the samples using the average teacher adjustment factor (TAF) from the 1990-91 weighting 
as a guideline. This adjustment factor is based on a weighted average of the ratio between 
the number of teachers reported on the school questionnaire (numerator) and the number of 
teachers reported on the TLR (denominator). The teacher adjustment factors for the public 
schools were at the state level, while the factors for the private schools were at the 
association membership (or affiliation) level (i.e., Catholic, Episcopal, Montessori, etc.). 

For public schools, each state’s TAF was defined as "good" if 0.9 < TAF < 1.1, and 
defined as "bad" if TAF > 1.1 or TAF < 0.9. For private schools, each affiliation’s TAF 
was defined as "good" if 0.8 < TAF < 1 and "bad" if TAF < 0.8. "Good" and "bad" 
refer to how similar the teacher counts were on both the school questionnaire and TLR. The 
"bad" states and affiliations are listed on the next page. 



Attachment A-2 
June 3, 1994 



Note : The private school TAFs were all less than 1 . This was because the weighted average 
of counts from the teacher file (i.e., TLR counts) were always greater than the 
weighted average of counts from the school file (i.e., school questionnaire). After the 
sample was selected, errors were found on the private teacher file which made those 
counts greater than they were supposed to be. 



"Bad’’, or Poor Reporting States 



The following states were poor reporting states, in terms of reporting consistent teacher 
counts in the 1987-88 and 1990-91 SASS using the school questionnaire and the TLR. 



• District of Columbia 

• Georgia 

• Illinois 

• Iowa 

• Massachusetts 

• Michigan 



• New Hampshire 

• Rhode Island 

• South Dakota 

• Vermont 

• Wisconsin 



"Bad", or Poor Reporting Association Memberships for Affiliations! 



• Catholic 

• Friends 

• Episcopal 

• National Society for Hebrew 
Day Schools 

• Solomon Schechter Day Schools 

• Other Jewish 



• Evangelical Lutheran Church in America 

• Other Lutheran 

• American Association of Christian Schools 

• National Association of Private Schools for 
Exceptional Children 

• Montessori 

• All Else 



The public school sample contained a higher percentage of schools from the poor (or "bad") 
reporting states (70% poor, 30% good). The private schools contained a higher percentage 
of schools from the poor reporting affiliations (75% poor, 25% good). This guaranteed us of 
getting differences in counts in (at least) some of the schools. 

In addition, schools from certain counties were dropped from the public school sample after 
sampling. These counties contain school districts that consistently had poor response rates 
(based on the 1987-88 and 1990-91 SASS). The counties are listed on the next page. 




3G 
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June 3, 1994 



Counties dropped from the Public School Sample 



• Baltimore MD 

• Chesterfield County VA 

• Dale County FL 

• Hartford CT 

• Howard County MD 

• Jersey City NJ 



• Madison WI 

• Montgomery County MD 

• Newark NJ 

• New Haven CT 

• Pinellas County FL 

• Richmond VA 
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Attachment B-l 
June 3, 1994 



Teacher / Non-teacher Categories for the TLVS Report 



Teachers 

(1) part-time (regardless of subject/title taught) 

Any teacher that was listed as part-time is included in this category, regardless of 
whether there was a subject listed. For example, if the respondent said that "we 
missed 3 part-time reading teachers", then they would be included in this category, 
since "part-time" was mentioned. 

(2) general full-time (no subject/title given) / general teacher (no explanation at all) 

For example, the schools simply reported that they "missed 3 teachers" or "we forgot 
to include a teacher". In other words, there was no explanation as to what kind of 
teachers they were (i.e., part-time, full-time, reading, math, etc.) 

(3) Subject Matter (math, english, science, social studies) / Kindergarten / Elementary 

(4) Specialized Subject Matter or Elective (PE/gym, music [band, orchestra, chorus/choir, 
instrumental], drama, vocational ed., industrial arts/tech, auto mechanics, carpentry, 
home ec., driver’s ed., art, typing, army/ROTC instructor, ELP [English Limited 
Proficiency], ESL/bilingual, foreign lang., reading/reading specialist, computers, 
religion, business ed., health, special services teacher) 

(5) Special Ed. (developmental reading, enrichment, basic skills, LD [learning disabled], 
IEU, behavior disorder, intermediate) / Gifted (exceptional, TAG) / College teacher for 
HS credit 

(6) Itinerant (any subject) / Migrant teacher / Co-op 

(7) Chapter 1 

(8) Long-term substitute (no subject/title given) 

(9) Other teacher (homebound, teacher employed by local govt., teacher employed by public 
school system that teaches in private school, non-paid/volunteers, non-certified teachers, 
teachers that teach in another building for this school, teachers on short-term leave [sick, 
personal day]) 
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Non-teachers 

(10) Principal / Asst. Principal / School Head / Dean of Students / President / other 
administrator / athletic director / coach 

(11) Guidance counselor / other counselor 

(12) Librarian / Media Specialist / Resource Person / Learning Center 

(13) Speech Therapist/Teacher/Pathologist 

(14) School Psychologist / Nurse / Physical Therapist 

(15) Other school staff (secretary, service staff, para-professional, social worker, in-school 
supervision, AV coordinator, planetarium keeper, caseworker, work coordinator, 
special ed. coordinator) 

(16) Pre-k / Pre-school / Director of Pre-school 

(17) Teacher Aides / student teacher / tutors (home tutor) / LSS (Learner Support 
Strategist) 

(18) Short-term substitute (no subject/title given) 

(19) Other non-teacher (teacher on long-term leave [sick, disability, maternity], teach in 
another school/teach in another school under this school’s principal, PE service that is 
not faculty, houseparents who teach their kids at home, different grade range on 
reinterview and original lead to difference, duplicate, teachers in school for different 
grade range than form, someone that gave lecture at school, teachers that teach at 
Tech. Institute in same building as school) 



Number 

94-01 

94-02 

94-03 

94-04 

94-05 

94-06 

94- 07 

95- 01 
95-02 
95-03 



Listing of NCES Working Papers to Date 



Title 



Schools and Staffing Survey (SASS) 
Papers Presented at Meetings of the 
American Statistical Association 

Generalized Variance Estimate for 
Schools and Staffing Survey (SASS) 

1991 Schools and Staffing Survey (SASS) 
Reinterview Response Variance Report 

The Accuracy of Teachers’ Self-reports 
on their Postsecondary Education: Teacher 
Transcript Study, Schools and Staffing 
Survey 

Cost-of-Education Differentials 
Across the States 

Six Papers on Teachers from the 
1990-91 SASS and Other Related 
Surveys 

Data Comparability and Public Policy: 
New Interest in Public Library Data 
Papers Presented at Meetings of the 
American Statistical Association 

Schools and Staffing Survey: 1994 
papers presented at the 1994 Meeting 
of the American Statistical 
Association 

QED Estimates of the 1990-91 Schools 
and Staffing Survey: Deriving and 
Comparing QED School Estimates with 
CCD Estimates 

Schools and Staffing Survey: 1990-91 
SASS Cross-Questionnaire Analysis 



Contact 



Dan Kasprzyk 
Dan Kasprzyk 
Dan Kasprzyk 



Dan Kasprzyk 
William Fowler 



Dan Kasprzyk 



Carrol Kindel 



Dan Kasprzyk 



Dan Kasprzyk 
Dan Kasprzyk 



Listing of NCES Working Papers to Date (Continued) 



Number 

95-04 

95-05 

95-06 

95-07 

95-08 

95-09 



Title 



National Education Longitudinal 
Study of 1988: Second Follow-up 
Questionnaire Content Areas and 
Research Issues 

National Education Longitudinal 
Study of 1988: Conducting Trend 
Analyses of NLS-72, HS&B, and 
NELS:88 Seniors 

National Education Longitudinal 
Study of 1988: Conducting Cross-Cohort 
Comparisons Using HS&B, NAEP, and 
NELS:88 Academic Transcript Data 

National Education Longitudinal 
Study of 1988: Conducting Trend Analyses 
HS&B and NELS:88 Sophomore Cohort 
Dropouts 

CCD Adjustments to the 1990-91 SASS: 

A Comparison of Estimates 

The Results of the 1993 Teacher List 
Validation Study (TLVS) 



Contact 



Jeffrey Owings 



Jeffrey Owings 



Jeffrey Owings 



Jeffrey Owings 
Dan Kasprzyk 
Dan Kasprzyk 
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